Movement monitoring system

ABSTRACT

A monitoring or tracking system may include an input port and a controller in communication with the input port. The input port may receive data from a data recorder. The data recorder is optionally part of the monitoring system and in some cases includes at least part of the controller. The controller may be configured to receive data via the input port and determine values for one or more dimensions of subject performing a task based on the data and determine a location of a hand of the subject performing the task based on the data. Further, the controller may be configured to determine one or both of trunk angle and trunk kinematics based on the received data. The controller may output via the output port assessment information.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional PatentApplication Ser. No. 62/932,802, filed on Nov. 8, 2019, the disclosureof which is incorporated herein by reference.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

This invention was made with government support under T42 OH008434 andR01 OH011024 awarded by the Center for Disease and Prevention. Thegovernment has certain rights in the invention.

TECHNICAL FIELD

The present disclosure pertains to monitoring systems and assessmenttools, and the like. More particularly, the present disclosure pertainsto video analysis monitoring systems and systems for assessing risksassociated with movement and exertions.

BACKGROUND

A variety of approaches and systems have been developed to monitorphysical stress on a subject. Such monitoring approaches and systems mayrequire manual observations and recordings, cumbersome wearableinstruments, complex linkage algorithms, and/or complexthree-dimensional (3D) tracking. More specifically, the developedmonitoring approaches and systems may require detailed manualmeasurements, manual observations over a long period of time, observertraining, sensors on a subject, and/or complex recording devices. Of theknown approaches and systems for monitoring physical stress on asubject, each has certain advantages and disadvantages.

SUMMARY

This disclosure is directed to several alternative designs for, devicesof, and methods of using monitoring systems and assessment tools.Although it is noted that monitoring approaches and systems are known,there exists a need for improvement on those approaches and systems.

Accordingly, one illustrative instance of the disclosure may include asubject tracking system. The subject tracking system may include aninput port and a controller in communication with the input port. Theinput port may receive data related to a subject performing a task. Thecontroller may be configured to determine a value for each of one ormore dimensions of the subject performing the task based on the data,determine a location of a hand of the subject performing the task basedon the data, and determine one or both of a trunk angle of the subjectperforming the task and one or more values of trunk kinematics of thesubject performing the task based on the value for at least onedimension of the one or more dimensions of the subject performing thetask and the location of the hand of the subject performing the task.

Alternatively or additionally to any of the embodiments above, the oneor more values of trunk kinematics of the subject performing the taskmay include a value of a velocity of movement of a trunk of the subjectperforming the task.

Alternatively or additionally to any of the embodiments above, the oneor more values of trunk kinematics of the subject performing the taskmay include a value of an acceleration of movement of a trunk of thesubject performing the task.

Alternatively or additionally to any of the embodiments above, the oneor more dimensions of the subject performing the task may include one orboth of a height dimension of the subject and a width dimension of thesubject.

Alternatively or additionally to any of the embodiments above, the oneor more dimensions of the subject performing the task may include awidth dimension of the subject.

Alternatively or additionally to any of the embodiments above, thelocation of the hand of the subject performing the task may include ahorizontal location of the hand of the subject and a vertical locationof the hand of the subject.

Alternatively or additionally to any of the embodiments above, wherein:the one or more dimensions of the subject performing the task mayinclude a width dimension of the subject, the location of the hand ofthe subject performing the task may include a horizontal location of ahand of the subject and a vertical location of the hand of the subject,and the controller is configured to use the following equation todetermine the trunk angle of the subject performing the task:

T=a+b*f(H)+c*f(V)+d*f(w),

where: a, b, c, and d are constants, H is a value of the horizontallocation of the hand of the subject performing the task, V is a value ofthe vertical location of the hand of the performing the task, w is avalue of the width dimension of the subject performing the task, and Tis a value of a trunk angle of the subject performing the task.

Alternatively or additionally to any of the embodiments above, whereinthe data related to the subject performing the task may include videodata and the controller may be configured to determine the one or moredimensions of the subject performing the task using pixel informationfrom the video data.

Alternatively or additionally to any of the embodiments above, thecontroller may be configured to automatically determine one or both ofthe trunk angle of the subject performing the task and the one or morevalues of trunk kinematics of the subject performing the task in realtime during playback of the video data.

Alternatively or additionally to any of the embodiments above, thecontroller may be configured to: identify a ghost effect in the videodata, the ghost effect having a location in a frame of the video data;and determine the location of the hand of the subject performing thetask based on the location of the ghost effect.

Alternatively or additionally to any of the embodiments above, whereinthe trunk angle may be one of a trunk flexion angle and a spine flexionangle.

Another illustrative instances of the disclosure may include a computerreadable medium having stored thereon in a non-transitory state aprogram code for use by a computing device, the program code causing thecomputing device to execute a method for determining one or both of atrunk angle of a subject and trunk kinematics of the subject. The methodmay include obtaining data related to the subject performing a task,determining a value for each of one or more dimensions of the subjectperforming the task based on the data, determining a location of a handof the subject performing the task based on the data, and determiningone or both of the trunk angle of the subject performing the task andone or more values of trunk kinematics of the subject performing thetask based on the value for at least one dimension of the one or moredimensions of the subject performing the task and the location of thehand of the subject performing the task.

Alternatively or additionally to any of the embodiments above,determining one or more values of trunk kinematics of the subjectperforming the task may include determining a velocity of movement of atrunk of the subject performing the task.

Alternatively or additionally to any of the embodiments above,determining one or more values of trunk kinematics of the subjectperforming the task may include determining an acceleration of movementof a trunk of the subject performing the task.

Alternatively or additionally to any of the embodiments above, the oneor more dimensions of the subject performing the task may include one ormore of a height dimension of the subject and a width dimension of thesubject.

Alternatively or additionally to any of the embodiments above, thelocation of the hand of the subject performing the task may include ahorizontal location of the hand of the subject and a vertical locationof the hand of the subject.

In another illustrative instance of the disclosure, a tracking systemmay include a processor and memory in communication with the processor.The memory may include instructions executable by the processor to:analyze pixel information in a video of a subject performing a task,determine a value for each of one or more dimensions of the subject in aframe from the video based on the pixel information, and determine atrunk angle of the subject in the frame based on the value for at leastone dimension of the one or more dimensions of the subject in the frame.

Alternatively or additionally to any of the embodiments above, thememory may include further instructions executable by the processor toautomatically determine trunk angles of the subject in real time duringplayback of the video.

Alternatively or additionally to any of the embodiments above, thememory may include further instructions executable by the processor todetermine one or both of a velocity of movement of a trunk of thesubject over a plurality of frames from the video and an acceleration ofmovement of the trunk of the subject over a plurality of frames from thevideo.

Alternatively or additionally to any of the embodiments above, the oneor more dimensions of the subject in the frame may include one or moreof a height dimension of the subject and a width dimension of thesubject.

Alternatively or additionally to any of the embodiments above, memorymay include further instructions executable by the processor to:identify a ghost effect in the frame, the ghost effect having a locationin the frame, determine a location of a hand of the subject in the framebased on the location of the ghost effect, determine extreme-most pixelsin a width dimension of the subject in the frame, assign a distancebetween the extreme-most pixels as a value of the width dimension, anddetermine the trunk angle of the subject in the frame based on the valueof the width dimension of the subject in the frame and the location ofthe hand of the subject in the frame.

The above summary of some example embodiments is not intended todescribe each disclosed embodiment or every implementation of thedisclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure may be more completely understood in consideration of thefollowing detailed description of various embodiments in connection withthe accompanying drawings, in which:

FIG. 1 is a schematic box diagram of an illustrative monitoring ortracking system;

FIG. 2 is a schematic box diagram depicting an illustrative flow of datain a monitoring system;

FIG. 3 is a schematic flow diagram of an illustrative method ofmonitoring movement of a subject;

FIG. 4 depicts a subject and an illustrative trunk angle of the subject;

FIG. 5 depicts a subject and illustrative parameters related to thesubject;

FIG. 6 is a schematic view of an illustrative monitoring systemcapturing images of a task being performed;

FIG. 7 is a schematic flow diagram of an illustrative method ofmonitoring movement of a subject;

FIG. 8A is an illustrative frame of video used as a reference frame in amonitoring system;

FIG. 8B is an illustrative frame of video that may be compared to thereference frame in FIG. 8A by the monitoring system;

FIG. 9 is a schematic view of an illustrative segmented frame of videodepicting a result from comparing the frame of video in FIG. 8B with thereference frame of FIG. 8A;

FIG. 10 is a schematic view of an illustrative segmented frame of videowith a bounding box applied around an identified subject;

FIGS. 11A-11C depict subjects in different illustrative orientations,where the subjects are bound by a bounding box;

FIG. 12 is a schematic diagram of an illustrative decision treetechnique for comparing values to thresholds;

FIGS. 13A-13E are schematic views of illustrative segmented frames ofvideo showing an illustrative ghost effect appearing and disappearing inthe frames of video over time;

FIG. 14 is a schematic view of an illustrative segmented frame of videodepicting a silhouette of a subject loading an object and in which afeet location of the subject is being determined;

FIG. 15 is a schematic view of an illustrative segmented frame of videodepicting a silhouette of a subject unloading an object and in which afeet location of the subject is being determined;

FIG. 16 is a schematic flow diagram of an illustrative method ofrepresenting portions of a subject in a frame of video; and

FIG. 17 is a schematic flow diagram of an illustrative method ofperforming a risk assessment.

While the disclosure is amenable to various modifications andalternative forms, specifics thereof have been shown by way of examplein the drawings and will be described in detail. It should beunderstood, however, that the intention is not to limit aspects of theclaimed disclosure to the particular embodiments described. On thecontrary, the intention is to cover all modifications, equivalents, andalternatives falling within the spirit and scope of the claimeddisclosure.

DESCRIPTION

For the following defined terms, these definitions shall be applied,unless a different definition is given in the claims or elsewhere inthis specification.

All numeric values are herein assumed to be modified by the term“about”, whether or not explicitly indicated. The term “about” generallyrefers to a range of numbers that one of skill in the art would considerequivalent to the recited value (i.e., having the same function orresult). In many instances, the term “about” may be indicative asincluding numbers that are rounded to the nearest significant figure.

The recitation of numerical ranges by endpoints includes all numberswithin that range (e.g., 1 to 5 includes 1, 1.5, 2, 2.75, 3, 3.80, 4,and 5).

Although some suitable dimensions, ranges and/or values pertaining tovarious components, features and/or specifications are disclosed, one ofskill in the art, incited by the present disclosure, would understanddesired dimensions, ranges and/or values may deviate from thoseexpressly disclosed.

As used in this specification and the appended claims, the singularforms “a”, “an”, and “the” include plural referents unless the contentclearly dictates otherwise. As used in this specification and theappended claims, the term “or” is generally employed in its senseincluding “and/or” unless the content clearly dictates otherwise.

The following detailed description should be read with reference to thedrawings in which similar elements in different drawings are numberedthe same. The detailed description and the drawings, which are notnecessarily to scale, depict illustrative embodiments and are notintended to limit the scope of the claimed disclosure. The illustrativeembodiments depicted are intended only as exemplary. Selected featuresof any illustrative embodiment may be incorporated into an additionalembodiment unless clearly stated to the contrary.

Physical exertion is a part of many jobs. For example, manufacturing andindustrial jobs may require workers to perform manual lifting tasks(e.g., an event of interest or predetermined task). In some cases, thesemanual lifting tasks may be repeated throughout the day. Assessing theworker's movements and/or exertions while performing tasks required bymanufacturing and/or industrial jobs and/or movements of workers inother jobs or activities may facilitate reducing injuries by identifyingmovement that may put a worker at risk for injury.

Repetitive work (e.g., manual work or other work) may be associated withmuscle fatigue, back strain, injury, and/or other pain as a result ofstress and/or strain on a person's body. As such, repetitive work (e.g.,lifting, etc.) has been studied extensively. For example, studies haveanalyzed what is a proper posture that reduces physical injury risk to aminimum while performing certain tasks and have also analyzed howmovement cycles (e.g., work cycles) and associated parameters (e.g., aload, a horizontal location of the origin and destination of the motion(e.g., a lift motion or other motion), a vertical location of the originand destination of the motion, a distance of the motion, a frequency ofthe motion, a duration of the movement, a twisting angle during themotion, a coupling with an object, etc.) relate to injury risk.Additional parameters associated with movement cycles that maycontribute to injury risk may include the velocity and acceleration ofmovement of the subject (e.g., velocity and/or acceleration of trunkmovement and/or other suitable velocity and/or acceleration ofmovement), the angle of a body of the subject (e.g., a trunk angle orother suitable angles of the body), and/or an object moved at an originand/or destination of movement. Some of these parameters may be used toidentify a person's risk for an injury during a task based on guidelinessuch as the National Institute for Occupational Safety and Health(NIOSH) lifting equation or the American Conference of GovernmentalIndustrial Hygienists (ACGIH) Threshold Limit Value (TLV) for manuallifting, among others. Additionally, some of these parameters have beenshown to be indicative of injury risk (e.g., risk of lower back pain(LBP) or lower back disorders (LBD), etc.), but are not typicallyutilized in identifying a person's risk for an injury during a task dueto it being difficult to obtain consistent and accurate measurements ofthe parameters.

In order to control effects of repetitive work on the body,quantification of parameters such as posture assumed by the body whileperforming a task, the origin and/or destination of objects liftedduring a task, duration of the task, position assumed during the task, atrunk angle assumed by the body while performing a task, kinematics ofthe body during the task, and frequency of the task, among otherparameters, may facilitate evaluating an injury risk for a workerperforming the task. A limitation, however, of identifying postures,trunk angles, trunk angle kinematics, the origin and destination ofmovement or moved objects, and/or analyzing movement cycles is that itcan be difficult to extract parameter measurements from an observedscene during a task.

In some cases, wearable equipment may be used to obtain and/or recordvalues of parameters in an observed scene during a task. Although thewearable equipment may provide accurate sensor data, such wearableequipment may require a considerable set-up process, may be cumbersome,and may impede the wearer's movements and/or load the wearer's body, andas a result, may affect performance of the wearer such that the observedmovements are not natural movements made by the wearer when performingthe observed task. Furthermore, it is difficult to identify an actualcontext of signals obtained from wearable instruments alone.

An example of commonly used wearable equipment is a lumbar motionmonitor (LMM), which may be used to obtained and/or record values ofparameters relating to movement of a subject performing a task. The LMMis an exoskeleton of the spine that may be attached to the shoulder andhips of a subject using a harness. Based on this configuration, the LMMmay provide reliable measurements of a position, velocity, andacceleration of the subject's trunk while the subject is performing atask. However, similar to other wearable equipment, the LMM may becostly, require a considerable set-up/training process, may imposeinterruptions to the wearer's regular tasks, etc. The burdens of wearingthe LMM and lack of other options for accurately measuring a trunk angleand/or trunk kinematics of a subject performing a task has led to safetyorganizations omitting trunk angle and/or trunk kinematics from injuryrisk assessments despite trunk angle and trunk kinematics beingassociated with work-related low-back disorders.

Observing a scene without directly affecting movement of the personperforming the task may be accomplished by recording the person'smovements using video. In some cases, complex 3D video equipment andmeasurement sensors may be used to capture video of a person performinga task.

Recorded video (e.g., image data of the recorded video) may be processedin one or more manners to identify and/or extract parameters from therecorded scene. Some approaches for processing the image data mayinclude recognizing a body of the observed person and each limbassociated with the body in the image data. Once the body and limbs arerecognized, motion parameters of the observed person may be analyzed.Identifying and tracking the body and the limbs of an observed person,however, may be difficult and may require complex algorithms andclassification schemes. Such difficulties in identifying the body andlimbs extending therefrom stem from the various shaped bodies and limbsmay take and a limited number of distinguishable features forrepresenting the body and limbs as the observed person changesconfigurations (e.g., postures) while performing a task.

Video (e.g., image data recorded with virtually any digital camera) of asubject performing a task may be analyzed with an approach that does notrequire complex classification systems, which results in an approachthat uses less computing power and takes less time for analyses than themore complex and/or cumbersome approaches discussed above. In somecases, this approach may be, or may be embodied in, a marker-lesstracking system. In one example, the marker-less tracking system mayidentify a contour, or a portion of a contour, of a subject (e.g., abody of interest, a person, an animal, a machine and/or other subject)and determine parameter measurements from the subject in one or moreframes of the video (e.g., a width dimension and/or a height dimensionof the subject, a location of hands and/or feet of a subject, a distancebetween hands and feet of the subject, when the subject is beginningand/or ending a task, and/or other parameter values). In some cases, abounding box (described in greater detail below) may be placed aroundthe subject and the dimension of the bounding box may be used fordetermining one or more parameter values and/or position assessmentvalues relative to the subject.

The data obtained from the above noted approaches or techniques forobserving and analyzing movement of a subject and/or other suitable datarelated to a subject may be utilized for analyzing positions and/ormovements of the subject and providing position and/or risk assessmentinformation of the subject using lifting guidelines, including, but notlimited to, the NIOSH Lifting Equation and the ACGIH TLV for manuallifting. The NIOSH Lifting Equation is described in greater detail inWaters, Thomas R. et al., “Revised NIOSH equation for the design andevaluation of manual lifting tasks”, ERGONOMICS, volume 36, No. 7, pages749-776 (1993), which is hereby incorporated by reference in itsentirety. Although the NIOSH and ACGIH equations are discussed herein,other equations and/or analyses may be performed when doing a riskassessment of movement based on observed data of a subject performing atask and/or otherwise moving including relating analyses that assessinjury risks based on values of a subject's trunk angle and/or trunkkinematics.

This disclosure discloses approaches for analyzing data related to asubject performing a task. The data related to a subject performing atask may be obtained through one of the above noted task observationapproaches or techniques and/or through one or more other suitableapproaches or techniques. As such, although the data analyzingapproaches or techniques described herein may be primarily describedwith respect to and/or in conjunction with data obtained from amarker-less tracking system, the data analyzing approaches or techniquesdescribed herein may be utilized to analyze data obtained with othersubject observation approaches or techniques.

Turning to the Figures, FIG. 1 depicts a schematic box diagram of amonitoring or tracking system 10 (e.g., a marker-less subject trackingsystem). The tracking system 10, as depicted in FIG. 1, may include acontroller 14 having a processor 16 (e.g., a microprocessor,microcontroller, or other processor) and memory 18. In some cases, thecontroller 14 may include a timer (not shown). The timer may be integralto the processor 16 or may be provided as a separate component.

The tracking system 10 may include an input port 20 and an output port22 configured to communicate with one or more components of or incommunication with the controller 14 and/or with one or more remotedevices over a network (e.g., a single network or two or more networks).The input port 20 may be configured to receive inputs such as data 24(e.g., digital data and/or other data from a data capturing deviceand/or manually obtained and/or inputted data) from a data recorder 23(e.g., an image capturing device, a sensor system, a computing devicereceiving manual entry of data, and/or other suitable data recorder),signals from a user interface 26 (e.g., a display, keypad, touch screen,mouse, stylus, microphone, and/or other user interface device),communication signals, and/or other suitable inputs. The output port 22may be configured to output information 28 (e.g., alerts, alarms,analysis of processed video, and/or other information), control signals,and/or communication signals to a display 30 (a light, LCD, LED, touchscreen, and/or other display), a speaker 32, and/or other suitableelectrical devices.

In some cases, the display 30 and/or the speaker 32, when included, maybe components of the user interface 26, but this is not required, andthe display 30 and/or the speaker 32 may be, or may be part of, a deviceor component separate from the user interface 26. Further, the userinterface 26, the display 30, and/or the speaker 32 may be part of thedata recording device or system 23 configured to record data 24 relatedto a subject performing a task, but this is not required.

The input port 20 and/or the output port 22 may be configured to receiveand/or send information and/or communication signals using one or moreprotocols. For example, the input port 20 and/or the output port 22 maycommunicate with other devices or components using a wired connection,ZigBee, Bluetooth, WiFi, IrDA, dedicated short range communication(DSRC), Near-Field Communications (NFC), EnOcean, and/or any othersuitable common or proprietary wired or wireless protocol, as desired.

In some cases, the data recorder 23 may be configured to record datarelated to a subject performing a task and may provide the data 24. Thedata recorder 23 may include or may be separate from the user interface26, the display 30, and/or the speaker 32. One or more of the datarecorder 23, the user interface 26, the display 30, the speaker 32and/or other suitable components may be part of the tracking system 10or separate from the tracking systems 10. When one or more of the datarecorder 23, the user interface 26, the display 30, and/or the speaker32 are part of the tracking system 10, the features of the trackingsystem 10 may be in a single device (e.g., two or more of the datarecorder 23, the controller 14, the user interface 26, the display 30,the speaker 32, and/or suitable components may all be in a singledevice) or may be in multiple devices (e.g., the data recorder 23 may bea component that is separate from the display 30, but this is notrequired). In some cases, the tracking system 10 may exist substantiallyentirely in a computer readable medium (e.g., memory 18, other memory,or other computer readable medium) having instructions (e.g., a controlalgorithm or other instructions) stored in a non-transitory statethereon that are executable by a processor (e.g., the processor 16 orother processor) to cause the processor to perform the instructions.

The memory 18 of the controller 14 may be in communication with theprocessor 16. The memory 18 may be used to store any desiredinformation, such as the aforementioned tracking system 10 (e.g., acontrol algorithm), recorded data 24 (e.g., video and/or other suitablerecorded data), parameters values (e.g., frequency, speed, acceleration,etc.) extracted from data, thresholds, equations for use in analyses(e.g., NIOSH Lifting Equation, ACGIH TLV for Manual Lifting, etc.), andthe like. The memory 18 may be any suitable type of storage deviceincluding, but not limited to, RAM, ROM, EEPROM, flash memory, a harddrive, and/or the like. In some cases, the processor 16 may storeinformation within the memory 18, and may subsequently retrieve thestored information from the memory 18.

As discussed with respect to FIG. 1, the monitoring or tracking system10 may take on one or more of a variety of forms and the monitoring ortracking system 10 may include or may be located on one or moreelectronic devices. In some cases, the data recorder 23 used with or ofthe monitoring or tracking system 10 may process the data 24 thereon.Alternatively, or in addition, the data recorder 23 may send, via awired connection or wireless connection, at least part of the recordeddata or at least partially processed data to a computing device (e.g., alaptop, desktop computer, server, a smart phone, a tablet computer,and/or other computer device) included in or separate from themonitoring or tracking system 10 for processing.

FIG. 2 depicts a schematic box diagram of the monitoring or trackingsystem 10 having the data recorder 23 connected to a remote server 34(e.g., a computing device such as a web server or other server) througha network 36. When so configured, the data recorder 23 may send recordeddata to the remote server 34 over the network 36 for processing.Alternatively, or in addition, the data recorder 23 and/or anintermediary device (not necessarily shown) between the data recorder 23and the remote server 34 may process a portion of the data and send thepartially processed data to the remote server 34 for further processingand/or analyses. The remote server 34 may process the data and send theprocessed data and/or results of the processing of the data (e.g., arisk assessment, a recommended weight limit (RWL), a lifting index (LI),etc.) back to the data recorder 23, send the results to other electronicdevices, save the results, and/or perform one or more other actions.

The remote server 34 may be any suitable computing device configured toprocess and/or analyze data and communicate with a remote device (e.g.,the data recorder 23 or other remote device). In some cases, the remoteserver 34 may have more processing power than the data recorder 23 andthus, may be more suitable for analyzing the data recorded by the datarecorder 23, but this is not always the case.

The network 36 may include a single network or multiple networks tofacilitate communication among devices connected to the network 36. Forexample, the network 36 may include a wired network, a wireless localarea network (LAN), a wide area network (WAN) (e.g., the Internet),and/or one or more other networks. In some cases, to communicate on thewireless LAN, the output port 22 may include a wireless access pointand/or a network host device and in other cases, the output port 22 maycommunicate with a wireless access point and/or a network access pointthat is separate from the output port 22 and/or the data recorder 23.Further, the wireless LAN may include a local domain name server (DNS),but this is not required for all embodiments. In some cases, thewireless LAN may be an ad hoc wireless network, but this is notrequired.

FIG. 3 depicts a schematic overview of an approach 100 for identifying,analyzing, and/or tracking movement of a subject based on received data.The approach 100 may be implemented using the tracking system 10, whereinstructions to perform the elements of approach 100 may be stored inthe memory 18 and executed by the processor 16. Additionally oralternatively, other suitable monitoring and/or tracking systems may beutilized to implement the approach 100.

The approach 100 may include receiving 110 data (e.g., the data 24and/or other suitable data) from a data source (e.g., the data recorder23 and/or other suitable source). Based on the received data, values ofone or more parameters related to a subject performing a task may bedetermined. For example, based on the received data, values of one ormore dimensions of a subject (e.g., while the subject performs a taskand/or at other suitable times) may be determined 112, one or morevalues indicative of a hand location of one or more hands of the subject(e.g., while the subject performs the task and/or at other suitabletimes) may be determined 114, and/or one or more other suitable valuesof parameters related to the subject performing the task may bedetermined. Based on values of the one or more dimensions of the subjectand/or values indicative of the hand location of the subject (e.g., thedetermined values of the one or more parameters of the subjectperforming a task), one or more values (e.g., location values, anglevalues, movement values, and/or other suitable values) related to atrunk position of the subject may be determined 116.

After determining values of one or more dimensions of the subject anddetermining values indicative of a location of the hands of the subject,parameters values, in addition to or as an alternative to parametervalues related to a trunk position of the subject, may be extracted fromthe received data to determine a position of the subject (e.g., alifting state or other state of the subject). In some cases, analyses ofthe values related to a trunk position of the subject, a position of thesubject, and/or other parameter values may include using the obtainedparameter values in the NIOSH Lifting Equation, the ACGIH TLVs forManual Lifting, equations evaluating risk of lower back pain (LBP),and/or other movement analyses equations to evaluate risk of injury tothe subject while performing a task recorded in the received data, butthe obtained parameter values may be analyzed for one or more otherpurposes.

The received data may be obtained or received 110 from any suitable datarecording component, device, or system. In some cases, the data may bereceived from a data recorder including, but not limited to, an imagecapturing device (as discussed in greater detail below), wearableequipment, manual measurements, sensors (e.g., photo sensors, infraredsensors, etc.), three-dimensional video equipment, and/or other suitabledevices configured to monitor and/or record data related to a subjectmoving or performing a task.

The received data may be any suitable type of data. For example, thedata may be digital data, analog data, video data, sensor data, manuallyentered data, and/or other suitable types of data.

Values of parameters may be extracted from the received data. In oneexample, values of one or more dimensions (e.g., a height, a width,etc.) of a subject may be determined 112 based on the received data inany suitable manner. In some cases, the received data may directlyindicate values of the one or more dimensions of the subject, mayindicate values of dimensions of a feature related to one or moredimensions of the subject, indicate values related to sensor readingsassociated with the subject, and/or indicate other suitable valuesrelated to one or more dimensions of the subject. In one example, valuesfor measurements of two or more sensors located on the subject at knownlocations may be extracted from the data and a difference between thevalues may be indicative of one or more dimensions of the subject. Inanother example, values for dimensions of a shape applied around thesubject may be extracted from the data and the one or more dimensions ofthe subject may be a function of the dimensions of the shape (e.g., awidth or height of a shape applied around a subject may be indicative ofa width or height of the subject). In a further example, values ofextreme-most points (e.g., extreme-most pixels when the data is videodata) of the subject in one or more dimensions of the subject may beextracted from the data and differences between extreme-most points ofthe subject in a dimension may be indicative of a value of the dimensionof the subject. Examples of extracting one or more dimensions of thesubject from received data based on shapes applied to the subject andextreme-most points of the subject are discussed below in greaterdetail.

The one or more dimensions of the subject may be any suitable type ofdimension of the subject. For example, the one or more dimensions of thesubject may be a height dimension of the subject, a width dimension ofthe subject, and/or other suitable dimension of the subject.

In another example of extracting values of parameters from the receiveddata, values related to or indicative of a hand location may bedetermined 114 based on the received data in any suitable manner. Insome cases, the hands of the subject may be initialized, recognized,and/or tracked manually. Alternatively or additionally, the hands of thesubject may be initialized, recognized, and/or tracked by software(e.g., in an automated manner) that analyzes the received data. In oneexample, the received data may provide values for measurements measuredby one or more sensors that may be associated with the hands of thesubject and the value indicative of the location of the hands may bedetermined based on the values for the measurements. In a furtherexample, one or more “ghost effects” in the data (e.g., video data) thatare indicative of an object being lifted or lowered by the subject(e.g., hence, also indicative of a location of the hands of the subject)may be identified and a location of the hands may be determined based ona location of the ghost effects. An example of determining valuesindicative of a hand location of the subject is discussed below ingreater detail and, although not necessarily required, may includeestimating a center of the ghost effect indicative of the object beinglifted or lowered by the subject.

Any suitable values indicative of a location of one or more hands of thesubject may be determined. In some cases, a value for a verticallocation of one or more hands of the subject and a value for ahorizontal location of one or more hands of the subject may bedetermined. In some cases, as discussed in greater detail below, a valueof a horizontal location of one or more hands of the subject may be adistance between a feet location of the subject and the hand location ofthe subject in a horizontal direction/dimension and a value of avertical location of one or more hands of the subject may be a distancebetween the feet location of the subject and the hand location of thesubject in a vertical direction/dimension. Alternatively oradditionally, a value of a horizontal location and/or a verticallocation of one or more hands of the subject may include horizontal andvertical pixel locations, respectively, on a pixel grid.

Values of, indicative of, or otherwise related to a trunk position ofthe subject may be determined 116 in any suitable manner based on thereceived data. In one example, the values related to or indicative of atrunk position of the subject may be determined based on one or morevalues of one or more dimensions of the subject and/or a value of a handlocation of the subject.

Example values of trunk positions of the subject that may be determinedbased on values of one or more dimensions of the subject and/or valuesindicative of a hand location of the subject include, but are notlimited to, a trunk angle of the subject, a velocity of movement of thetrunk of the subject, an acceleration of movement of the trunk of thesubject, and/or other suitable positional values related to the trunk ofthe subject. FIG. 4 depicts a subject 2 having a trunk angle, A_(T). Thetrunk angle, A_(T) may be defined by an imaginary line or plane, P_(T),extending through a spine of the subject 2 if the spine of the subject 2were straight and a line or plane, P_(vertical), perpendicular to a lineor plane, P_(horizontal), of or parallel to a surface supporting thesubject 2. When determining a trunk angle and/or trunk kinematics, adistinction may be made between a trunk flexion angle, a spine flexionangle, and/or other suitable trunk angles. A trunk flexion angle may bethe trunk angle, A_(T), as depicted in FIG. 4, where the subject 2 isbending forward and the line or plane, P_(T), that extends through aspine of the subject 2 also extends through a hip of the subject 2. Aspine flexion angle may be an angle of a line or plane that extendsthrough a spine of a subject and the L5/S1 disc of the spine when thesubject is bending forward at the L5/S1 disc.

Additionally or alternatively, values of one or more dimensions of thesubject and/or values indicative of a hand location of the subject maybe utilized to determine or estimate other suitable parameters relatedto the subject performing a task and/or the task including, but notlimited to, a horizontal hand location of the subject relative to feetof the subject, a vertical hand location of the subject relative to thefeet of the subject, a displacement of a bounding box upper-left corner,a displacement of a bounding box upper-right corner, a ratio of a heightof a bounding box to a width of the bounding box, an angle between abounding box diagonal and a bottom of the bounding box, a length of abounding box diagonal, an angle between a line segment between hands andfeet of the subject and the floor, a length of a line segment betweenhands and feet of the subject, a posture of the subject, a velocity ofmovement of the subject, an acceleration of movement of the subject,and/or values of one or more other suitable parameters of the subjectthat may assist in assessing injury risk to a subject performing a task.

Example suitable parameters that may be determined or estimated fromvalues of one or more dimensions of the subject and/or the hand locationof the subject include, but are not limited to, those depicted in anddescribed with respect to Table 1:

TABLE 1 Parameters Description Equation H Hand horizontal locationrelative to feet Lifting monitor algorithm V Hand vertical locationrelative to feet Lifting monitor algorithm BH Bounding box heightLifting monitor algorithm BW Bounding box width Lifting monitoralgorithm DUL Displacement of bounding box upper- Lifting monitor leftcorner algorithm DUR Displacement of bounding box upper- Lifting monitorright corner algorithm R Ratio of BH to BW $R = \frac{BH}{BW}$ DA Anglebetween bounding box diagonal and the bottom${DA} = {\arctan\;\frac{BH}{BW}}$ DL Bounding box diagonal length DL ={square root over (BH² + BW²)} HDA Angle between the line segmentbetween hands and feet and the floor ${HDA} = {\arctan\;\frac{H}{W}}$HDL Length of the line segment between HDL = {square root over (H² +V²)} hands and feetFIG. 5 depicts the subject 2 within a bounding box 44, to provide avisual description of the BH, BW, DA, DL, HAD, and HDL. Illustrativelifting monitor algorithms used to determine some of the parameters ofTable 1 are described below, for example, with respect to FIGS. 6-17.Further, Table 1 and an illustrative lifting monitor algorithm used todetermine some of the parameters in Table 1 are described in Greene R.L., Lu M. L., Barim M. S., Wang X, Hayden M., Hu Y. H., Radwin R. G.,Estimating Trunk Angle Kinematics During Lifting Using a ComputationallyEfficient Computer Vision Method. Hum Factors, 2020 Sep.24:18720820958840. doi: 10.1177/0018720820958840. Epub ahead of print.PMiD: 32972247, which is incorporated herein by reference in theirentirety. Additionally or alternatively, suitable parameters other thanthose depicted in Table 1 may be determined or estimated from values ofone or more dimensions of the subject and/or the hand location of thesubject are contemplated.

Generally, a trunk angle of the subject may be determined or estimatedfrom one or more of a value of a height measurement (h) of a subject(e.g., estimated by a height of a bounding box, BH, applied around thesubject and/or determine in one or more other suitable manners), a valueof a width measurement (w) of a subject (e.g., estimated by a width of abounding box, BW, applied around the subject and/or determine in one ormore other suitable manners), a ratio of the value of the heightmeasurement (h) to the value of the width measurement (w) at, a value ofa hand vertical location (V) of the subject, a value of a handhorizontal location (H) of the subject, a value of a ratio of a handvertical location (V) to the value of the hand horizontal location (H)of the subject, and/or values of one or more other suitable parametersrelated to the subject and/or task being performed.

Values related to a trunk position (e.g., trunk angle, trunk kinematics,etc.) of the subject may be determined 116 from data received by fittingone or more equations to the data. Illustrative techniques for measuringand/or determining trunk kinematics (trunk angle, trunk velocity, andtrunk acceleration) are described herein and in Greene et al.,Estimating Trunk Angle Kinematics During Lifting Using a ComputationallyEfficient Computer Vision Method, (2020), which was incorporated byreference above.

In some cases, a trunk angle (e.g., a trunk flexion angle, a spineflexion angle, and/or suitable trunk angle) of the subject may bedetermined using the following general equation:

T=a+b*f(H)+c*f(V)+d*f(w)+e*f(h)+f*f(H/V)+g*f(h/w)  (1)

where a, b, c, d, e, f, and g are constants and T is the trunk angle ofthe subject. In some cases, one or more constants a, b, c, d, e, f, andg may be equal to zero, but this is not required. For example, it may befound that the following equation provides an accurate estimate of atrunk angle:

T=a+b*f(H)+c*f(V)+d*f(w)  (2)

where the constants e, f, and g in equation (1) are all equal to zero.

One example application of equation (1) for determining or estimatingtrunk angles (e.g., a trunk flexion angle) of a subject may be:

T=76.42−2.14*ln(H)−1.12*(ln(H))²−7.97*ln(V)−1.32*(ln(V))²+0.16*BW  (3).

A further example application of equation (1) for determining orestimating trunk angles (e.g., a spine flexion angle) of a subject maybe:

T=85.63−2.12*(ln(H))2−5.34*ln(V)−0.62*(ln(V))2+0.58*BW  (4)

Such example applications of equation (1) and/or other suitableapplications of equation (1) may be selected independent of values of orstates of one or more parameters of the subject performing the task.Other suitable applications of equation (1) are contemplated.

In some cases, an application of equation (1) may be selected based on avalue of one or more parameters of the subject performing the task. Forexample, a state and/or value of one or more parameters of the subjectperforming the task at time, t, may be determined and then, based on thedetermined state and/or value of the one or more parameters, anapplication of equation (1) may be selected to determine a trunk anglevalue or other trunk position value of the subject at the time, t.

In some cases, the one or more parameters of the subject performing thetask may be or may include a posture of the subject, where the states ofthe posture may be determined and an application of equation (1) orother suitable equations may be selected based on the determined stateof the posture. In some cases, the states of the posture of the subjectperforming the task may include, but are not limited to, a standingposture, a stooping posture, and a squatting posture. The posture stateof the subject may be determined using video data analysis techniquesdescribed in greater detail below, video data analysis techniques otherthan those described herein, analyses of non-video data related toposture states of the subject, and/or other suitable analyses of datarelated to the subject performing a task. Alternatively or in addition,a manually entered posture state may be utilized. In one example ofdetermining a trunk angle of a subject performing a task based on aposture state of the subject, an application of equation (1) when thesubject is determined to be in a standing state may be:

T=−55.40*V ²−71.07*(h/w)+66.89  (5);

an application of equation (1) when the subject is determined to be in astooping state may be:

T=−55.40*V ²−71.07*(h/w)+66.89  (6);

an application of equation (1) when the subject is determined to be in asquatting state may be:

T=−48.67*H+26.18*H ²−150.31*V−51.27*V ²+64.10*w+73  (7).

Other suitable applications of equation (1) based on postures states ofthe subject and/or based on other states and/or values of other suitableparameters of the subject may be utilized.

As discussed above, values of trunk position parameters of a subject maybe determined 116 from the received data in addition to values of atrunk angle. For example, values of trunk kinematics including, but notlimited to, values of a trunk velocity of the subject (e.g., a value ofa velocity of movement of a trunk of the subject performing a task),values of a trunk acceleration of the subject (e.g., a value of anacceleration of movement of a trunk of the subject performing a task),and/or values of other suitable trunk-related parameters of the subjectmay be determined from received data. In one example, a trunk velocitymay be determined by taking an absolute value of a difference between atrunk position at a first time determined from an application ofequation (1) and a trunk position at a second time determined from anapplication of equation (1) and then, dividing by an absolute value of adifference between the first time and the second time (e.g., adifference in time, frames, relative times of frames, etc.). Similarly,a trunk acceleration may be determined by taking an absolute value of adifference between a trunk velocity at a first time and a trunk velocityat a second time and then, dividing by an absolute value of a differencebetween the first time and the second time.

In another example of fitting one or more equations to the data todetermine a value related to a trunk position, an exponential equationmay be fitted to the received data. In some cases, linear or nonlinearregression or other suitable analyses may be used to determine thefitted equation. Fitting an equation, such as an exponential equation,to trunk angles (e.g., trunk angles during a lift) may facilitateestimating trunk angles over a series of consecutive video frames (e.g.,dynamically), along with calculating trunk kinematics (e.g., trunkspeed/velocity, acceleration, etc.) over a series of video frames.

Illustratively, such an exponential equation may take the followinggeneral form:

T=α·e ^(βx)  (8)

where T, is a trunk angle of a subject during a lift of an object at aframe number, x, of a series of frames (e.g., a series of consecutive ornon-consecutive frames) depicting a subject lifting the object and α, βare coefficients determined based on the received and/or calculated data(e.g., H, V, BH, BW, R, etc.) related to the subject lifting the object.The variables for determining or predicting the coefficients, α, β, ofthe exponential equation may include, but are not limited to, theaverage, maximum, and standard deviations for features in the receivedand/or calculated data, along with the respective speed and accelerationover the set of frames depicting the subject performing the lift. Insome cases, the coefficients, a, β, may differ based on a posture orpositioning of the subject (e.g., whether the subject is stooping,squatting, or standing), but this is not required.

In one example, one or more illustrative exponential equations modelinga trunk angle over multiple frames depicting a lift by the subject maybe determined from values of and/or derived from H, V, BH, BW, and R(e.g., as described in Table 1 above) over 20 frames of a subjectperforming a lift. In some cases, the one or more illustrativeexponential equations may be determined for stoop lifts, squat lifts,and standing lifts. Although other techniques may be utilized,regression models may be used to determine or predict coefficients α, βof equation (8) for stoop lifts, squat lifts, and standing lifts. Asummary of the regression models used to determine or predictcoefficients α, β of equation (8) for stoop lifts, squat lifts, andstanding lifts is provided in Table 5:

TABLE 5 Variables Estimates CI p Coefficient (α) stoop (Intercept) 2.37 −1.48-6.22   0.222 SD DA speed 14.79    3.45-26.14   0.012* Mean DLacceleration −0.07  −0.18-0.04   0.198 SD DL speed 0.07  −0.03-0.16  0.161 Mean HDA speed −4.71  −8.43-−0.99   0.014* Mean DA acceleration−0.91  −1.78-−0.05   0.039* Max DA acceleration −0.13  −0.22-−0.04  0.004** SD DL acceleration 0    0.00-0.00 <0.001*** Mean HDAacceleration −0.33  −0.69-0.03   0.069 SD HDA acceleration 0.04 −0.02-0.10   0.148 Mean HDL acceleration −0.01  −0.01-0.00   0.1 DUR−0.02  −0.05-0.01   0.143 squat (Intercept) 13.54    4.39-22.69   0.01**SD DA acceleration −0.24  −0.53-0.05   0.088 Mean DL acceleration 0.02   0.00-0.04   0.048* SD DL acceleration 0  −0.00-0.00   0.066 Max HDAacceleration −0.1  −0.19-−0.00   0.046* SD HDL acceleration 0.2 −0.06-0.47   0.111 stand (Intercept) 1.37  −2.80-5.54   0.514 Mean DLspeed 0.06  −0.00-0.12   0.062 Max HDA speed 0.06    0.03-0.10 <0.001***SD HDA speed −7.42 −15.51-0.66   0.071 Mean HDA speed 0.17  −0.05-0.38  0.137 SD HDA speed −0.31  −0.52-−0.10   0.004** Mean DA acceleration1.11  −0.26-2.48   0.109 Mean DL acceleration 0.01    0.00-0.02   0.016*Max HDL acceleration 0.09    0.01-0.16   0.021* DUL −0.02  −0.05-0.01  0.18 Coefficient (β) stoop (Intercept) 0.1    0.05-0.15 <0.001*** MeanDA acceleration 1.57e⁻²    0.00-0.03   0.034* Max DA acceleration2.26e⁻³    0.00-0.00   0.035* SD DA acceleration 4.36e⁻³  −0.01-0.00  0.082 Mean DL acceleration 9.40e⁻⁵  −0.00-0.00   0.063 SD DLacceleration −1.65e⁻⁵  −0.00-−0.00   0.025* squat (Intercept) −0.02 −0.11-0.06   0.544 Mean DA acceleration 3.52e⁻²    0.01-0.06   0.012*Mean DL acceleration −2.94e⁻⁴  −0.00-−0.00   0.012* SD DL acceleration4.21e⁻⁵    0.00-0.00   0.009** Max HDA acceleration 2.21e⁻³    0.00-0.00  0.002** SD HDA acceleration −5.34e⁻³  −0.01-−0.00   0.004** stand(Intercept) 1.00e⁻¹    0.05-0.15 <0.001*** Note: SD refers to standarddeviation. *p < .05. **p < 0.01. ***p < .001.The coefficient β when the subject is performing a stand lift is aconstant (0) due to utilizing a minimum flexion when performing astanding lift.

Trunk speed and/or acceleration of the trunk of the subject during thelift may be determined in any suitable manner. In one example, based onthe determined trunk angles and a video frame rate of the frames of thesubject performing the lift, trunk speed and acceleration in each framemay be calculated. Alternatively or additionally, the velocity andacceleration of trunk movement may be determined mathematically bytaking derivatives of the fitted functions.

FIGS. 6-17 depict systems and approaches for identifying, analyzing,and/or tracking movement of a subject based on received data, where thedata is video data and the system may implement an application of theapproach 100. In some cases, the video data may be obtained in anon-intrusive manner (e.g., without the use of sensors). Other systemsand approaches for identifying, analyzing, and/or tracking movement of asubject based on received data are contemplated.

FIG. 6 is a schematic view of an image capturing device 12 (e.g., whichmay be an example of the data recorder 23 and/or other suitable datarecorder) of, or used in conjunction with, a tracking and/or monitoringsystem (e.g., a tracking or monitoring or tracking system 10, as shownin FIG. 1), where the image capturing device 12 is set up to observe thesubject 2 perform a task (e.g., moving objects 4 from a shelf 6 to atable 8, as shown in FIG. 6, or other task). The image capturing device12 may be or may include one or more of a phone (e.g., a camera-equippedsmart phone or other phone), a portable digital camera, a dedicateddigital camera (e.g., a security camera or other dedicated camera), adigital camcorder, a tablet computer, a laptop, a desktop, and/or asuitable other electronic device capable of recording video.

Any suitable number of image capturing devices 12 may be utilized. Insome cases, video of the subject 2 performing a task may be capturedwith two or more image capturing devices 12. Additionally oralternatively, although 2D data is primarily discussed herein as beingcaptured by the image capturing device(s) 12, the image capturingdevice(s) 12 may be utilized to capture 3D image data of the subject 2and such 3D image data may be utilized to analyze a task performed bythe subject 2 in a manner similar to those described herein for captured2D video data.

FIG. 7 depicts a schematic overview of an approach 200 for identifyingand analyzing movement of a subject (e.g., the subject 2 or othersuitable subject) in video without the use of sensors or continuoustracking of limbs of a subject via linkage algorithms. In some cases,the approach 200 may be considered an application of the approach 100,but this is not required.

The approach 200 may include receiving 202 video from an image capturingsource (e.g., the image capturing device 12 or other suitable datarecorder) and identifying 204 the subject in the video. Once the subjectis identified 204, the subject may be bound 206 and the hands of thesubject may be located 208. After locating 208 the hands of the subject,parameters values extracted from the video based on the identifiedsubject, the bound of the subject, the location of the hands of thesubject, and/or other parameters may be analyzed 210 to determine aposition of the subject (e.g., a lifting state, a posture state,kinematics, a trunk angle, trunk kinematics, and/or other state orposition of the subject). In some cases, the analyses may include usingthe obtained parameter values in the NIOSH Lifting Equation, the ACGIHTLVs for Manual Lifting, an equation determining injury risk (e.g.,lower back disorder injury risk and/or other suitable injury risks)based on a trunk position of the subject, and/or other movement analysisequations to evaluate risk of injury to the subject while performing atask recorded in the video, but the obtained parameter values may beanalyzed for one or more other purposes.

Identifying 204 the subject in received video may be accomplished in oneor more manners. For example, the subject may be identified 204 inreceived video by manually identifying the subject and/or by identifyingthe subject in an automated or at least partially automated manner(e.g., automatically and/or in response to a manual initiation). Asubject may be manually identified by manually outlining the subject, byapplying a shape (e.g., a box or other shape) around the subject, byclicking on the subject, and/or manually identifying the subject in oneor more other manners. Background subtraction or other suitabletechniques may be utilized to automatically identify or identify in anautomated manner a contour of the subject (e.g., a foreground). Othersuitable manual techniques and/or automated techniques may be utilizedto identify a subject in received video.

Background subtraction may be performed in one or more manners. Ingeneral, background subtraction may be performed by statisticallyestimating whether a pixel in the current frame of video (e.g., eachpixel or a set of pixels in the current frame) belongs to the backgroundor the foreground depicted in the frame. To facilitate statisticallyestimating whether a pixel belongs to the background or the foregrounddepicted in a frame, each pixel or set of pixels may be given a valuebased on a feature (e.g., color, shading, intensity, etc.) of the pixel.Here, an underlying assumption is that values of a background pixel in avideo will change slowly over time (e.g., background pixels may beexpected to remain unchanged for at least a plurality of consecutiveframes of video) compared to values of a foreground pixel (e.g.,foreground pixels, especially those on or around a periphery of asubject, may be expected to change from frame-to-frame in video and/orat least more rapidly than background pixels). As a result, values of apixel over a fixed window of a past set of frames can be used toestimate the pixel value at the current frame (e.g., in some cases, theestimated pixel value may be considered an expected pixel value). If theprediction is sufficiently accurate with respect to an actual pixelvalue at the current frame, this pixel is likely to be and/or may beconsidered to be a background pixel. Otherwise, this pixel is likely tobe and/or may be considered to be a foreground pixel. Alternatively orin addition, an estimated pixel value may be indicative of a foregroundpixel and if the prediction is sufficiently accurate with respect to anactual pixel value at the current frame, the pixel is likely to beand/or may be considered to be a foreground pixel. Otherwise, the pixelis likely to be and/or may be considered to be a background pixel.

As used herein, a pixel may be a smallest addressable element in animage or display device. Each pixel used to depict a frame of video mayhave an address or physical coordinates in a two-dimensional grid in theframe.

One may model the values of a pixel over a fixed number of past videoframes using a Mixture of Gaussian (MOG) model and update the modelparameters adaptively as the algorithm progresses over time to provideestimates of pixel values and determine if a pixel belongs to thebackground or the foreground. An example MOG approach is described inZivkovic, Zoran. “Improved adaptive Gaussian mixture model forbackground subtraction.” Pattern Recognition, 2004, ICPR 2004,Proceedings of the 17th International Conference on. Vol. 2. IEEE, 2004,which is hereby incorporated by reference in its entirety. Anotherexample MOG approach is described in Zivkovic, Zoran, and Ferdinand VanDer Heijden. “Efficient adaptive density estimation per image pixel forthe task of background subtraction.” Pattern recognition letters 27.7(2006): 773-780, which is hereby incorporated by reference in itsentirety. Additionally, or alternatively, other modeling techniquesand/or segmentation approaches may be utilized to differentiate betweena background and a foreground.

FIGS. 8A and 8B depict frames of a video. In FIG. 8A, a frame having abackground 38 is depicted without a subject 2. FIG. 8B depicts a framehaving the subject 2 with the background 38 of or substantially similarto the background 38 in FIG. 8A. One of the frame in FIG. 8A and theframe in FIG. 8B may be considered a reference frame and pixels of theother frame may be compared to corresponding possible pixel values froma distribution developed based on at least the reference frame and eachpixel in the other frame may be assigned an indicator of beingbackground or foreground (e.g., a number value, a color (e.g., black orwhite), etc.) using the segmentation approaches discussed above.

FIG. 9 depicts a foreground subtraction resulting from segmenting FIG.8B relative to FIG. 8A. As this may be the beginning of the video, thebackground may change and the possible background pixel value Gaussiandistribution mixture model (e.g., the MOG model or other model) may beof only one distribution component with a mean value of the distributionbeing the same as the pixel values in FIG. 8A. The appearance of themoving subject in FIG. 7B may not be matched into the correspondingbackground model and as a result, the pixels of the moving subject maybe considered the foreground (e.g., as represented in white as asilhouette 40) and the rest of the pixels may be considered thebackground (e.g., as represented in black). Although the segmentation isdepicted in FIG. 9 with the background being black and the foregroundbeing white, other colors or techniques (e.g., outlining, etc.) may beused to distinguish between a foreground and a background.Alternatively, segmentation may not be depicted and a display may depictthe original video during and/or after processing of the video or novideo at all.

Although the background in the frames of FIG. 8A and FIG. 8B is staticor substantially static, the background subtraction techniques describedabove may be utilized on dynamically changing backgrounds. In such casesand/or in other suitable instances, an initialization of the subject 2may be done to distinguish the moving subject 2 from other movingobjects in the background. Such initialization may be accomplished bymanually or automatically applying a bounding box (e.g., as discussedbelow) to or around the subject and/or may be accomplished in one ormore other manners. After the initialization of the subject 2, anyobjects identified as moving (e.g., through identifying a ghost effectblob) between frames may be compared to the initialized subject 2 in aprevious frame and only moving objects matching the initialized subject2 in the previous frame will be kept as foreground or as the subject 2.

In some cases, the monitoring or tracking system 10 may not be able torecognize an entirety of the subject 2, which may result in anincomplete silhouette 40 of the subject 2 (e.g., the silhouette may haveone or more holes or gaps 42, as shown in FIG. 9) being produced fromcomparing pixels of successive frames of video. Such holes or gaps 42may appear due to noise in the environment (e.g., illumination changes,shadows, etc.) around the background 38 and/or due to a pixelrepresenting part of the subject 2 (e.g., one or more pixels in theframe) that may have an appearance (e.g., intensity value) that is closeto that of a pixel of the background 38 in a reference frame, such thatthe pixel value matches the background model. It is contemplated thatthe holes or gaps 42 may occur in a silhouette for one or more otherreasons.

The holes or gaps 42 in a silhouette 40 may be addressed in one or moremanners. In one example, the holes or gaps 42 may be filed throughmorphological and/or other techniques that fill-in gaps betweenidentified portions of the silhouette 40.

Once the subject 2 has been identified in the video by identifying thesilhouette 40 and/or by one or more other suitable technique, thesubject 2 may be bound 206, as depicted in the illustrative approach 200of FIG. 7. The subject 2 may be bound 206 using one or more manualand/or automated techniques.

In one example of bounding the subject 2, marginal pixels of thesilhouette 40 of the subject 2 in a horizontal direction and in avertical direction may be identified. That is, an extreme-most pixel ofthe silhouette 40 in a positive y-direction, an extreme-most pixel ofthe silhouette 40 in the negative y-direction, an extreme-most pixel ofthe silhouette 40 in a positive x-direction, and an extreme-most pixelof the silhouette 40 in a negative x-direction may be identifiedrelative to a center of the silhouette 40. A height dimension of thesilhouette 40 may be identified by taking a difference of a verticalcoordinate location on the grid of the frame for the extreme-most pixelof the silhouette 40 in the positive y-direction and a verticalcoordinate location on the grid of the frame for the extreme-most pixelof the silhouette 40 in the negative y-direction. A width dimension ofthe silhouette 40 may be identified by taking a difference of ahorizontal coordinate location on the grid of the frame for theextreme-most pixel of the silhouette 40 in the positive x-direction anda horizontal coordinate location on the grid of the frame for theextreme-most pixel of the silhouette 40 in the negative x-direction. Theheight dimension and the width dimension of the silhouette 40 may beused as or assigned as a height dimension and width dimension,respectively, of the subject 2.

Alternatively, or in addition, the subject 2 may be bound 206 byapplying a bounding box 44 around silhouette 40, as shown in FIG. 10.The bounding box 44 may be applied close around the silhouette 40. Insome cases, an edge of the bounding box 44 may tangentially pass each ofthe marginal pixels of the silhouette 40 in a positive y-direction, anegative y-direction, a positive x-direction, and a negative x-directionrelative to a center of the silhouette 40. Alternatively or in addition,the bounding box 44 may be applied around the silhouette 40 to bound thesubject 2 so as to extend through one or more other pixels of thesilhouette 40 and/or the background 38. The height and width dimensionsof the bounding box 44 may be equal to or may be indicative of a heightdimension and width dimension of the silhouette 40. Similar to asdiscussed above, the height dimension and/or the width dimension of thebounding box 44 may be used as or assigned as a height dimension and awidth dimension, respectively, of the subject 2. Further, in some cases,the height and width dimensions of the bounding box 44 may be indicativeof an object 4 location, a hand location of the subject 2, a posturestate of the subject 2, a trunk angle of the subject 2, and/or otherparameter values.

FIGS. 11A-11C depict the subject 2 in three different postures,respectively, with a bounding box 44 and identified locations ofmarginal pixels. In FIG. 11A, the subject 2 is in a standing position orposture, in FIG. 11B the subject 2 is in a stooping position or posture,and in FIG. 11C the subject 2 is in a squatting position or posture.Although other postures of the subject may be identified, a standingposture, a stooping posture, and a squatting posture (e.g., see FIGS.11A-11C for depictions of these postures) may be a focus of postureanalysis due to the relevance of these postures in the NIOSH liftingequation and/or the ACGIH TLV for manual lifting, among other analyses.

Each of FIG. 11A, FIG. 11B, and FIG. 11C depict a coordinate system 46relative to a center of a height and width of the subject 2. Thecoordinate system 46 is depicted for descriptive (e.g., relational)purposes only and is not necessarily part of the monitoring or trackingsystem 10. Further, FIGS. 11A-11C depict a marginal pixel 48 in thex-positive direction, a marginal pixel 50 in the x-negative direction, amarginal pixel 52 in the y-positive direction, and a marginal pixel 54in the y-negative direction. Then, optionally, the bounding box 44 maybe applied around the subject 2 (or the silhouette 40 of the subject 2)tangential to or otherwise relative to the marginal pixels 48, 50, 52,54.

As can be seen from FIGS. 11A-11C, each of the respective orientationsor postures of the subject 2 correspond to a different height and widthdimension of the subject 2 or bounding box 44. It has been found that aheight and a width of a subject 2 or bounding box 44 correlates with anorientation (e.g., posture, trunk angle, or other orientation) of thesubject 2 and/or other parameters relative to the subject 2. As such, ina lifting analysis, the height and width dimension of the subject 2 orthe bounding box 44 may be utilized to determine or predict at least theorientation of the subject 2 and to determine injury risks for thesubject without complex linkage algorithms, sensors and sensor data, andmanual measurements (e.g., hip and/or knee angles, etc.). Theorientation(s) of the subject and/or injury risks in view thereof may bedetermined in real time (e.g., in real time during recording of videoand/or during playback of video), but this is not required.

In some cases, a posture of the subject may be determined from thesubject's identified and/or assigned dimensions based on one or more ofthe dimensions of the bound in one or more suitable manners. In oneexample, the posture of the subject may be determined based on thesubject's determined height dimension and the subject's determined widthdimension, but other dimensions of the subject may be additionallyand/or alternatively used.

As referred to above, a technique for identifying standing, stopping,and squatting postures of a subject in frames of video that does notrequire tracking of bending angles of a subject in video, use ofsensors, use of manual identification, or use of complex limb trackingalgorithms may include determining these posture states based on one orboth of the subject's height and width dimensions. In some cases, avalue based on the height dimension of the subject and a value based onthe width dimension of the subject (e.g., the height and widthdimensions of the subject, normalized height and width dimensions of thesubject, and/or other values based on the height and width dimensions ofthe subject) may be compared to one or more height thresholds and one ormore width thresholds, respectively, to determine the subject's posture.

When determining whether a subject in a frame of video is in a standingposture, a stooping posture, or a squatting posture, the value based onthe height of the subject may be compared to a first height thresholdand a second height threshold and the value based on a width of thesubject may be compared to a first width threshold and a second widththreshold. The values based on the dimensions of the subject may becompared to thresholds using one or more suitable techniques and in oneor more suitable orders. In some cases, a technique using a decisiontree 250, as shown in FIG. 12 for example, may be utilized. Additionalor alternative techniques for comparing the values based on thedimensions of the subject to thresholds may include, but are not limitedto, look up tables, algebraic equations, and/or other suitabletechniques.

In an example using a decision tree, such as when the decision tree 250is utilized, an analysis may start at a top of the decision tree 250 andinitially the value based on the height of the subject may be comparedto the first height threshold H_(TH1). When the value based on theheight of the subject has reached or gone beyond (e.g., is equal to orgreater than) the first height threshold H_(TH1), the subject may beclassified as being in and/or may be assigned a standing posture. Whenthe value based on the height of the subject has not reached or gonebeyond the first height threshold H_(TH1), then the value based on theheight of the subject may be compared to the second height thresholdH_(TH2) When the value based on the height of the subject has notreached or gone beyond (e.g., is less than) the second height thresholdH_(TH2), the subject may be classified as being in and/or may beassigned a squatting posture. When the value based on the height of thesubject has not reached or gone beyond the first height thresholdH_(TH1) and has reached or gone beyond the second height thresholdH_(TH2), a value based on the width of the subject may be compared tothe first width threshold W_(TH1). When the value based on the width ofthe subject has reached or gone beyond the first width thresholdW_(TH1), the subject may be classified as being in and/or may beassigned a stooping posture. When the value based on the height of thesubject has not reached or gone beyond the first height thresholdH_(TH1) and has reached or gone beyond the second height thresholdH_(TH2) and the value based on the width of the subject to has notreached or gone beyond the first width threshold W_(TH1), the valuebased on the width of the subject may be compared to the second widththreshold W_(TH2). When the value based on the width of the subject hasreached or gone beyond the second width threshold W_(TH2), the subjectmay be classified as being in and/or may be assigned a standingposition. When the value based on the width of the subject has notreached or gone beyond the second width threshold W_(TH2), the subjectmay be classified as being in and/or may be assigned a squattingposition. Using a decision tree technique for comparing values baseddimensions to dimension threshold values may result in an efficientdetermination of posture information for a monitored subject.

Values of the threshold values related to height and width of thesubject may be suitable values related to a type of value that is usedfor the values based on the height and width dimensions of the subject.For example, when the values based on the height and width dimensionsare normalized using a normalizer of a standing height of a subject inthe example above, the first height threshold H_(TH1) may be a value ina range from about 0.90 to about 0.95, the second height thresholdH_(TH2) may be a value in a range from about 0.63 to about 0.68, thefirst width threshold W_(TH1) may be a value in a range from about 0.64to about 0.69, and the second width threshold W_(TH2) may be a value ina range from about 0.51 to about 0.57. In one example, the first heightthreshold H_(TH1) may be a about 0.93, the second height thresholdH_(TH2) may be about 0.66, the first width threshold W_(TH1) may beabout 0.67, and the second width threshold W_(TH2) may be about 0.54.Other values for thresholds used in determining a posture of a subjectare contemplated.

In addition to or as an alternative to being able to extract postureinformation and/or other information from video to assess injury risk orfor another purpose, it may be useful to be able to locate 208 the handsof the subject, as presented in the approach 200 of FIG. 7. Handlocation may be determined in any suitable manner. In some cases, thehands of the subject may be initialized, recognized, and/or trackedmanually or by software (e.g., in an automated manner)

A technique has been developed to identify the hands of the subject 2during at least frames of when a task starts and frames of when a taskends. In some cases, such a technique may utilize identifying “ghosteffects” when the subject 2 loads and/or unloads the object 4 and maynot require training of a continuous hand detector and may be able avoidor mitigate error due to difficulties in differentiating hands fromother portions of the subject 2. This technique of identifying hands orlocations of hands of the subject 2 using ghost effects may be used inaddition to or as an alternative to other suitable techniques foridentifying hands of the subject, including, but not limited to,techniques utilizing manual identification of hands, techniques forcontinuous or semi-continuous (e.g., at specified intervals) trackingover time, techniques utilizing a continuous hand detector, and/or othersuitable techniques for identifying hands or locations of hands of thesubject 2.

A ghost effect may be a set of connected and/or adjacent points (e.g., aset of pixels in a frame of video data) detected as being in motion, butnot corresponding to any real moving objects. Such a definition of“ghost effects” is discussed in Shoushtarian, B. and Bez, H. “Apractical adaptive approach for dynamic background subtraction using aninvariant colour model and object tracking.” Pattern RecognitionLetters, January 2005, 26(1):5-26, January 2005, which is herebyincorporated by reference in its entirety. For example, a ghost effectmay be a cluster of pixels that represents an appearance of a staticobject or a region of a scene where these pixels look different in acurrent frame than in one or more immediately previous frames. The ghosteffect may appear and then disappear into background after thebackground model learns and updates the new appearance of these pixelsover a plurality of frames.

As such, in some cases, the ghost effects may be considered to be aby-product of the background subtraction technique discussed above andmay be utilized to identify when a task begins and/or ends, along with alocation of the hands of the subject when the task begins and/or ends.For example, as the background subtraction technique may update thebackground model (e.g., the Gaussian distribution background model, MOG)over two or more frames to adapt for backgrounds that are not static, itmay take several frames for a moving object 4 to be consideredbackground after the moving object 4 stops (e.g., becomes static) and/oris separated from the subject 2. Similarly for a static object 4 thatstarts to move, the location where the object 4 was may take severalframes to be considered background. As a result of this delay inrecognizing what is background and what is foreground, a location of amoving object 4 after it stops moving (e.g., an ending location) and/ora location of a static object 4 before it begins moving (e.g., abeginning location) may show up as a blob or ghost effect in a frame ofvideo.

As discussed, one case in which a ghost effect may occur is when astatic object is moved and values of pixels at the region where theobject was static become different from estimated values of the pixelbased on a background model for pixels at the region and thus, thatregion may be considered to be foreground and/or depicted as such in aframe. The background model may then take several (e.g., two or more)frames to learn the new static appearance of that region and absorb thepixels of that region into the background model. That is, before thebackground model updates, the pixels of the region where the object wasstatic are labeled as foreground and are considered to depict a ghosteffect.

As discussed, another case where a ghost effect might occur is when amoving object becomes static. A region where the object stops may changeits appearance from a previous appearance when the object was notpresent (e.g., the background) into an appearance associated with asubject or moving object (e.g., the foreground). As the background modelof the region is built up with only pixel values for the previousappearance for when the object was not present in the region, a newpresence of the static object in the region may be considered to beforeground. The background model may then take several frames to learnthe new static appearance of the region with the newly received objectand absorb the pixels of that region into the background model. Beforethe background model updates, the pixels of the region where the objectstopped moving may be labeled as foreground and/or may be considered aghost effect.

Further and as discussed in greater detail below, the ghost effects 56,as shown for example in FIGS. 13A-13E, 14, and 15, may be detected, anda subject's hand location may be determined from the detected ghosteffects 56, by looking for clusters of foreground pixels in a frame ofvideo that were not present in a reference frame or frames of the video.In some cases, ghost effects 56 may be identified when clusters ofpixels satisfy certain principles. The principles may include, amongother principles, consistency in time (e.g., a cluster of pixels show upin the same location in the following N frames), gradual vanishing(e.g., a size of a cluster should be no larger than a size of the objectand may gradually become smaller over a set of frames), the subject 2 isin close proximity to the cluster of pixels when the cluster of pixelsare initially identified, and a number of frames it may take for acluster of pixels to become background is consistent with an expectednumber of frames for the ghost effect 56 to disappear. To be considereda ghost effect 56, a cluster of pixels may need to satisfy one, some, orall of the above referenced principles and/or other principles.

The monitoring or tracking system 10 may search for an object appearingon a portion of the frame (e.g., the ghost effect 56 of the object 4).In some cases, if it is known that a task begins on a left side or otherportion of a frame of video, the monitoring or tracking system 10 maylook for the object or ghost effect appearing in the left side or theother portion of the frame. Similarly, if it is known that a task endson a right side or other portion of the frame, the monitoring ortracking system 10 may look for the object or ghost effect appearing inthe right side or the other portion of the frame. If it is not knownwhere in a frame a task is expected to begin and/or end, the monitoringor tracking system 10 may look for the object or ghost effect in theentire frame.

FIGS. 13A-13E depict example frames with the subject 2 and the ghosteffect 56 as the ghost effect 56 is first identified and fades away overtime. FIG. 13A depicts a frame with the ghost effect 56 near in time towhen the ghost effect initially appears and the subject 2 picks up theobject 4. Ten (10) frames after the frame of FIG. 13A, the frame of FIG.13B depicts the ghost effect 56 being separate from the subject 2 andobject 4. Ten (10) frames after the frame of FIG. 13B, the frame of FIG.13C depicts the ghost effect 56 starting to fade into the background.Ten (10) frames after the frame of FIG. 13C, the frame of FIG. 13Ddepicts the ghost effect 56 almost completely faded into the background.Ten (10) frames after the frame of FIG. 13D, the frame of FIG. 13E nolonger depicts the ghost effect 56. Although FIGS. 13A-13E depict theghost effect 56 completely or substantially disappearing into thebackground after fifty (50) frames, the ghost effect 56 may beconfigured to be depicted for a longer or shorter amount of time (e.g.,in more or fewer frames).

As the subject's 2 hands may be at the location of the ghost effect 56to move the object 4 at the beginning of a task (e.g., the location ofthe ghost effect 56 in FIG. 13A) and at the location of the ghost effect56 to place the object 4 at the ending of a task (e.g., the location ofthe ghost effect 56 in FIG. 13D), a hand location of the subject 2 maybe determined (e.g., inferred) from the location of the ghost effects56. The ghost effect 56 may initially occur at a beginning of task(e.g., when an object 4 starts to move) and/or at an end of a task(e.g., when an object 4 first becomes stationary and separated from thesubject 2), a first frame in which the ghost effect 56 is identified(e.g., a first frame in a sequence of frames in which the ghost effect56 appears) and a position of the ghost effect 56 in the first frame maybe recorded as the time of a beginning or ending of a task and alocation of the hands of the subject 2 at that time, respectively.Further, the ghost effect 56 may have a shape (e.g., form a region ofinterest about a hand of the subject 2) when it initially appears in thevideo and the shape of the ghost effect 56 at this point may be tracked(e.g., a center of the shape of the ghost effect 56 and/or othersuitable portion of the ghost effect 56 may be tracked) from thebeginning of the task to the ending of the task to determine locationsof the ghost effect 56, and thus the locations of the hands of thesubject 2, over performance of the task by the subject 2. Alternativelyor additionally, the locations of the hands of the subject 2 at a timeduring the task may be determined from a function of or algorithm basedon a hand location of the subject 2 at the beginning of the task, a handlocation of the subject 2 at the end of the task, one or more dimensionsof the subject 2 at the beginning of the task, one or more dimensions ofthe subject 2 at the ending of the task, one or more dimensions of thesubject 2 during the task, and/or one or more other suitable parameters.

When a location of the object 4 is tracked during the performance of atask by the subject 2 via tracking a shape of the object 4, the object 4may blend in with one or more portions of foreground at one or moretimes during performance of the task. For example, this may occur whenthe subject 2 switches from facing a first direction to facing a seconddirection and/or may occur at other times. Losing a location of theobject 4 during the task may be problematic because a location of theobject 4 may indicate or imply a location of the hands of the subject 2during the performance of the task. Although the object 4 may blend inwith other portions of the foreground during the task, temporalinformation and/or appearance information may be utilized to extrapolateor interpolate position information concerning the object 4 when theshape of the object 4 cannot be identified after it has been initiallyrecognized.

Additionally or alternatively to identifying the beginning and ending ofa task based on identifying ghost effects and although not required, adetermination of the frame(s) where the task may start and/or end may bebased at least partially on information known about a task. For example,as it may be known that the subject 2 or a portion of the subject 2performing a repetitive task reverses direction after starting and/orending the task, a start and an end of a task may be initiallyidentified or confirmed by tracking a horizontal location of the subjectin the frames of the video.

Once the locations of the hands of a subject 2 during a beginning and/oran ending of a task are identified, a vertical and/or horizontaldistance between the locations of the hands and a location of the feetof the subject 2 may be determined. When the monitoring or trackingsystem 10 is performing a task analysis, such as a lifting analysis ortrunk angle analysis, the vertical and horizontal distances between thefeet and hands when loading and unloading an object may be necessary tocalculate a recommended weight limit, a trunk angle, trunk kinematics(e.g., trunk speed, trunk acceleration, etc.) and/or may be utilized bythe monitoring or tracking system 10 to perform other analyses. In somecases, the vertical and horizontal distances between the feet and handsof the subject may be considered examples of vertical and horizontal,respectively, locations of the hand of the subject 2.

Although the monitoring or tracking system 10 may determine a handlocation within a frame as discussed above and/or in one or more othersuitable manners, a location of the feet within the frame(s) of videomay need to be determined to provide a relative positioning of thehands. The vertical location of the feet may be considered to be thesame as the base of the bounding box (e.g., a margin pixel in thenegative y-direction). The horizontal coordinate of the feet locationmay be determined in one or more manners including, but not limited to,by using a weighted sum of a horizontal silhouette pixel index. Thehorizontal silhouette pixel index is, for example:

$\begin{matrix}{{{Feet}\mspace{14mu}{Center}_{horizontal}} = \frac{\sum_{i = {mostleftpixelindex}}^{mostrightpixelindex}{i \times {weight}_{i}}}{\sum_{i = {mostleftpixelindex}}^{mostrightpixelindex}{weight}_{i}}} & (9)\end{matrix}$

The weight_(i) may be the total number of pixels that is covered by thesilhouette 40 at corresponding horizontal index i.

Before applying the above formula, however, the monitoring or trackingsystem 10 may need to determine a region of interest where the feetcenter may lie. This may be entered manually through a user interface orthe monitoring or system 10 may determine, on its own, the region ofinterest where the feet center lies. In one example, the monitoring ortracking system 10 may set the region of interest where the feet centerlies as an area of the subject's feet and shanks (e.g., shins) asrepresented by the silhouette 40. FIGS. 14 and 15 depict the region ofinterest 58 of a silhouette 40 bounded with a bounding box 44 fordetermining a center of the feet when the silhouette 40 is beginning atask by loading an object represented by ghost effect 56 (FIG. 14) andwhen the silhouette 40 is ending a task by unloading the objectrepresented by the ghost effect 56 (FIG. 15). The monitoring or trackingsystem 10 may then determine the mass center of this area using equation(9).

The shank and feet area (e.g., the region of interest) may be determinedin any manner. In one example, a statistical method may be used to findthe height of the shanks of the subject 2 as represented by thesilhouette 40. For example, a shank height may be considered to be apercentage of a total height of the subject. In some cases, the shankheight may be considered to be 0.10, 0.15, and/or other fraction of aheight of the silhouette 40 of the subject 2. Thus, a vertical dimensionof the region of interest where the feet center may lie may span from0.15 of a height of the silhouette 40 of the subject 2 in the frame anda vertical dimension of the base of the bounding box 44. The horizontaldimension of the region of interest may span from a marginal pixel ofthe silhouette 40 in a x-positive direction (e.g., the right-most pixelindex) within the vertical dimension of the region of interest and amarginal pixel of the silhouette 40 in a x-negative direction (e.g., themost left pixel index) within the vertical dimension of the region ofinterest 58, as depicted in FIG. 14.

In the situation where the subject 2 may be squatting and working withan object 4 near the ground, as shown in FIG. 15, it is contemplated thehands of the silhouette 40 representing the subject 2 and/or the ghosteffect 56 representing the object 4 may be located in the region ofinterest 58. To facilitate determining a horizontal location of the feetwhen the hands and/or the object are located in the region of interest58, the region of interest 58 may be adjusted (e.g., horizontallyreduced) based on a size of the object 4 as represented by the ghosteffect 56. The size of the object 4 may be determined by multiplying adistance from the center of the ghost effect 56 (e.g., which may havebeen determined to locate the hands) to an edge of the bounding box 44by two (2), as the outer edge of the object 4 may typically beconsidered a margin pixel defining an edge of the bounding box 44.

Once the region of interest 58 is identified, a distance between thehands and feet of the subject 2 may be determined. The distance betweenthe hands and feet of the subject 2 may then be used to determine atrunk angle, assess movement, perform risk assessments, and/or performother suitable analyses of the subject 2 in the video.

Although segmentation of frames of video facilitates identifyingsubjects 2 and objects 4 in video based on movement of the subjects 2and/or objects 4 relative to a background, one limitation is that if oneof the subjects 2 and/or objects 4 of interest, or portions thereof,stops moving for a set number of frames (e.g., a predetermined number oftwo or more frames, which may depend on a background update rate), thatsubject 2 and/or object 4, or portions thereof, may be absorbed into thebackground. As a result, features that are meant to be in the foregroundand be identified or tracked may become background and untrackable.

In one instance of features meant to be in the foreground but thatbecome background, among others, feet may be stationary at one or moretimes while the subject 2 is performing a monitored task and thus, thefeet may be absorbed into the background. Losing a location of the feetmay be problematic because a useful measurement in monitoring thesubject 2 performing a task is a horizontal and/or vertical distancebetween the subject's hands and feet, as discussed above. Additionallyor alternatively, when the feet disappear from a foreground in segmentedvideo, a bound around the subject 2 may change and the subject 2 may beassigned an inaccurate posture and/or other parameter measurements maybe affected. Although the feet disappear from the silhouette 40representing the subject 2 due to a lack of motion of the feet, temporalinformation and/or appearance information may be utilized to retrievethe feet and maintain the feet in the foreground when the feet arestationary.

To account for the situation when feet and/or other portions of asilhouette representing a subject disappear from the foreground when itis desirable for such portions of the silhouette to be in theforeground, a location of the feet and/or other portions of the subjectmay be identified by utilizing an estimation of the location of thefeet. For example, a location of the feet and/or other portions of thesubject in a previous frame and/or a function thereof may be added toand/or substituted into a current frame when the feet and/or otherportions of the subject have disappeared from the foreground in thecurrent frame. In some cases, Bayesian-based estimation may be utilizedto ensure the foreground in each frame of video includes a silhouette ofthe feet of the subject. Although we discuss estimating locations of thefeet of a subject, other portions of the subject may be located throughestimation in a manner similar to as discussed herein with respect tothe feet of the subject.

One example formula that may be used to estimate a location of thesubject's feet is as follows:

Posterior probability=prior probability×likelihood  (10)

where the prior probability term in equation (10) may be an estimatedfeet location of a silhouette 40 in a current frame of video based on aprevious frame of video. In one example, the estimated feet location maybe the location of the feet of the silhouette 40 (e.g., region ofinterest 58 or other location) in the previous frame or a differentfunction of the location of the feet of the silhouette 40 in theprevious frame. Because the feet may not move fast from frame-to-framefor a conventional video frame rate (e.g., a frame rate in a range from15 frames per second (fps) to 30 fps), the difference between the feetlocation of a silhouette 40 in the current frame and that of theprevious frame may be expected to be small (e.g., as measured in changeof pixel locations from frame-to-frame), with an average of about zero(0) pixels. As such, a plausible location for a feet portion of thesilhouette 40 in the current frame may be defined by one or more pixelsextending from the feet location of the silhouette 40 in a previousframe. As discussed above, the region of interest 58 may identify aplausible location of the feet of the subject 2 represented by thesilhouette 40 in the current frame. This region of interest 58 may bethe bottom 10% of the area covered by the bounding box of the previousframe (the area shares the same width and 0.1 of the height of thebounding box) or other suitable percentage of the area covered by thebounding box of the previous frame.

The likelihood term in equation (10) may be provided by motioninformation and appearance information. In some cases, the motioninformation and the appearance information may be weighted relative toeach other, but this is not required. In one example, the appearanceinformation may have a greater weight (e.g., have a higher priority)than the motion information. To compare a current frame appearance inthe region of interest 58 with a previous frame appearance within theregion of interest 58 (e.g., where the previous frame appearance withinthe region of interest 58 may be from a frame immediate before thecurrent frame, may be from a frame at X-number of frames before thecurrent frame, may be an average of previous frame appearances withinthe region of interest 58 over X-number of frames, a rolling average ofprevious frame appearances within the region of interest 58 overX-number of frames, or other suitable previous frame appearance withinthe region of interest 58), a pixel-by-pixel intensity cross-correlationof the region of interest 58 of the current frame and of the region ofinterest 58 of the previous frame may be utilized. If a confidence valueof the cross-correlation (e.g., a confidence level obtained as a directresult of the pixel-by-pixel intensity cross-correlation as compared toan expected result of the pixel-by-pixel intensity cross-correlation)goes beyond (e.g., is greater than, as depicted in FIG. 16, or lessthan) a pre-set confidence threshold (e.g., the pre-set threshold may beset as 0.85, 0.9, 0.95, and/or set at one or more other suitablethreshold values), then the feet portion of the silhouette in the regionof interest 58 of the current frame may be estimated to be the same asthat of the previous frame and the region of interest 58 of the previousframe may be utilized for the region of interest 58 of the currentframe. This happens when the feet of the subject are steady and themotion information is disregarded. If the confidence value ofcross-correlation has not reached (e.g., is lower than, as depicted inFIG. 16, or is greater than) the pre-set confidence threshold, then thefeet portion of the silhouette 40 in the region of interest 58 for thecurrent frame may be utilized as the feet of the silhouette 40 in thecurrent frame. This happens when the feet of the subject 2 are movingand motion information is considered.

The pixel-by-pixel intensity cross-correlation and confidence leveldetermination of the region of interest 58 in different frames of videomay be performed using digital image correlation and tracking (DIC)techniques and/or other suitable cross-correlation techniques. Althoughcross-correlation and the use of confidence levels is discussed hereinfor comparing the regions of interest 58 in a current frame and aprevious frame of video to determine feet locations of the subject 2,other comparing techniques may be utilized to determine locations offeet of the subject 2 and/or locations of other features of the subject2.

Returning to the illustrative approach 200 depicted in FIG. 7, once thesubject 2 has been bound 206 (e.g., by a bounding box or extreme-mostpixels) to determine one or more dimensions of the subject 2 and handsof the subject 2 have been located 208 (e.g., where the location of thehands may include a vertical location being a vertical distance betweenhands and feet of the subject and a horizontal location being ahorizontal distance between hands and feet of the subject), values ofsuch parameters extracted from the received video may be analyzed 210.In some cases, such parameters may be analyzed by applying values of theextracted parameters to an application of equation (1), equation (8),and/or other suitable equation to determine trunk angle and/or trunkkinematic values, the NIOSH Lifting Equation, the ACGIH TLV for manuallifting, and/or other analysis tools for assessing injury risk and/ormitigating injury of a subject performing a task. In one example, anapplication of equation (1) (e.g., equation (3), equation (4), and/orother suitable application of equation (1)) may result in a trunk angleand/or trunk kinematic determining function that is based on a value ofa width dimension of the subject 2 (e.g., as measured by a bounding boxand/or extreme-most pixels of the subject 2 in the width dimension) anda hand location of the subject 2 (e.g., a vertical location of a hand ofthe subject 2 and a horizontal location of the hand of the subject 2).In the example, the values of the trunk angle and/or trunk kinematicsmay provide and/or may be utilized to determine injury risk assessmentsfor a subject performing a task.

FIG. 16 depicts a flow diagram of an approach 300 for ensuring feet ofthe subject 2 appear in the silhouette 40 even when the subject's feetare or are substantially static or stationary across a predeterminednumber of frames. In the approach 300, a current frame may be providedfor analysis and background subtraction. As shown in a motioninformation box 310, background subtraction may be performed 312 on theframe to obtain 314 a segmented frame, including a region of interest 58for the feet of the subject 2 represented by the silhouette 40. As shownin an appearance information box 320, a cross-correlation for the regionof interest 58 in the segmented frame of the current frame and theregion of interest 58 in a segmented frame of a previous frame may beobtained and a confidence level in the correlation may be determined andcompared 324 to a confidence level threshold C_(TH). In the casedepicted in FIG. 16, the confidence level in the cross-correlation maybe determined by comparing the result of the cross-correlation to anexpected cross-correlation indicating the feet of the subject 2 asrepresented by the silhouette 40 have disappeared from the foreground inthe current frame, but other arrangements for determining a confidencelevel are contemplated. When the determined confidence level has gonebeyond (e.g., is greater than) the confidence level threshold C_(TH),the region of interest 58 and/or a portion of the silhouette 40 withinthe region of interest 58 of the segmented previous frame may besubstituted 326 into the segmented current frame and the final segmentedframe may be obtained 328. When the determined confidence level has notgone beyond (e.g., is equal to or less than) the confidence levelthreshold C_(TH), the region of interest 58 in the segmented currentframe may be the obtained 328 final segmented frame. This technique forensuring that stationary feet of the subject 2 appearing in a segmentedframe when the feet have not moved or have not moved a threshold amountover a predetermined number of frames is an example technique, and othertechniques are contemplated.

FIG. 17 is an approach 400 utilizing the monitoring or tracking system10 to assess movement of a subject during an event of interest (e.g., alifting task or other event of interest). Although not shown, theapproach may include receiving a video including an event of interest.The monitoring or tracking system 10 (e.g., a non-transitory computerreadable medium having instructions stored thereon to perform thetechniques discussed herein) may compare 402 pixels in frames of videoto possible pixel values based on an identified distribution to identifya subject within the frames of the video. As discussed above, themonitoring or tracking system 10 may compare successive frames of thevideo by comparing corresponding pixels of the successive frames and/orby comparing the frames in one or more other manners. Once the subjecthas been identified, a beginning of an event of interest and an endingof the event of interest may be determined 404 (e.g., by identifyingghost effects and/or with one or more other techniques). The event ofinterest may be any event involving the subject. In some cases, theevent of interest may be a lifting task that is repeated over time. Thetechniques discussed herein and/or other techniques may be utilized todetermine a beginning and/or an ending of an event of interest. One ormore coordinates (e.g., marginal pixels, center of pixel mass, etc.) ofa subject within a frame may be tracked 406 through a plurality offrames of the video as the subject moves within the frames over a periodof time from the beginning of the event of interest and the end of theevent of interest. When the event of interest involves a lifting task,the subject may be tracked from a location at which an object is pickedup (e.g., loaded) until a location at which the object is set down(e.g., unloaded). Further, if the event of interest is repeated, thesubject may be tracked while the event of interest is repeated. Then,based on coordinates of the subject during the event of interest andextracted information based on the identified coordinates (e.g.,including, but not limited to, one or more dimensions of the subject,hand locations of the subject, posture states of the subject, etc., asdiscussed herein), the monitoring or tracking system 10 may perform 408an assessment (e.g., a risk assessment) of movement of the subjectduring the event of interest.

In some cases, the monitoring or tracking system 10 may identify orextract parameter values from the video including, but not limited to,frequency (e.g., from the horizontal location tracking), speed (e.g., anamount of time between a beginning of an event and an end of the event),acceleration, and/or other parameter of the subject during the event ofinterest. Based on these parameters, posture, distance between hands andfeet of the subject, and/or other parameters, the monitoring or trackingsystem 10 may determine a trunk angle of the subject, trunk kinematicsof the subject, a recommended weight limit, a lifting index, and/orperform one or more other assessments (e.g., injury risk assessmentsand/or other suitable assessments) of movements of the subject duringthe event of interest. The monitoring or tracking system 10 may thenprovide an output (e.g., an alert, report, etc.) in response to theassessment and/or save the assessment to memory. Further, the monitoringor tracking system 10 may be configured to capture and/or receive videoin real time during an event of interest and perform real timeprocessing and/or assessments, in accordance with the approach 500 andas discussed herein, with the goal of preventing injuries and/ormitigating injury risks during the event of interest. In some cases,“real time” may include during recording of video and/or during playbackof video.

Further, during the process of the monitoring or tracking system 10processing the video, the video may be converted to frames similar to asdepicted in FIGS. 9, 10, 13A-13E, 15, and 16, where the background andthe foreground have been distinguished, and displayed on a display(e.g., the display 30 or other display) for observation while themonitoring system analyzes the video. Alternatively, the original videomay be displayed and the comparison of corresponding pixels insuccessive frames may be done in a process that is not displayed.Further, one or more of the bounding step and the hand location step(e.g., marking of an identified center of the hands) may be depicted ona display even if the comparison of corresponding pixels in successiveframes is not depicted in a manner similar to what is depicted in FIGS.9, 10, 13A-13E, 15, and 16, but rather the original video is displayedif any video is displayed. In some cases, the monitoring or trackingsystem 10 may output via the output port 22 assessments and/or alertsbased on assessments without displaying a portion of, or any portion of,an analysis of the video.

Although the monitoring or tracking system 10 is discussed in view ofmanual lifting tasks, similar disclosed concepts may be utilized forother tasks involving movement. Example tasks may include, but are notlimited to, manual lifting, sorting, typing, performing surgery,throwing a ball, etc. Additionally, the concepts disclosed herein mayapply to analyzing movement of people, other animals, machines, and/orother devices.

Further discussion of monitoring or tracking systems, techniquesutilized for processing data, and performing assessments (e.g., injuryrisk assessments) is found in U.S. patent application Ser. No.15/727,283 filed on Oct. 6, 2017, and is titled MOVEMENT MONITORINGSYSTEM, now U.S. Pat. No. 10,482,283, which is hereby incorporated byreference in its entirety, U.S. patent application Ser. No. 16/038,664filed on Jul. 18, 2018, and is titled MOVEMENT MONITORING SYSTEM, nowU.S. Pat. No. 10,810,414, which is hereby incorporated by reference inits entirety, and U.S. patent application Ser. No. 16/874,883 filed onMay 15, 2020, and is titled MOVEMENT MONITORING SYSTEM, which is herebyincorporated by reference in its entirety.

Those skilled in the art will recognize that the present disclosure maybe manifested in a variety of forms other than the specific embodimentsdescribed and contemplated herein. Accordingly, departure in form anddetail may be made without departing from the scope and spirit of thepresent disclosure as described in the appended claims.

What is claimed is:
 1. A subject tracking system comprising: an inputport for receiving data related to a subject performing a task; acontroller in communication with the input port, the controller isconfigured to: determine a value for each of one or more dimensions ofthe subject performing the task based on the data; determine a locationof a hand of the subject performing the task based on the data; anddetermine one or both of a trunk angle of the subject performing thetask and one or more values of trunk kinematics of the subjectperforming the task based on the value for at least one dimension of theone or more dimensions of the subject performing the task and thelocation of the hand of the subject performing the task.
 2. The systemof claim 1, wherein the one or more values of trunk kinematics of thesubject performing the task include a value of a velocity of movement ofa trunk of the subject performing the task.
 3. The system of claim 1,wherein the one or more values of trunk kinematics of the subjectperforming the task include a value of an acceleration of movement of atrunk of the subject performing the task.
 4. The system of claim 1,wherein the one or more dimensions of the subject performing the taskinclude one or both of a height dimension of the subject and a widthdimension of the subject.
 5. The system of claim 1, wherein the one ormore dimensions of the subject performing the task include a widthdimension of the subject.
 6. The system of claim 1, wherein the locationof the hand of the subject performing the task includes a horizontallocation of the hand of the subject and a vertical location of the handof the subject.
 7. The system of claim 1, wherein: the one or moredimensions of the subject performing the task include a width dimensionof the subject; the location of the hand of the subject performing thetask includes a horizontal location of a hand of the subject and avertical location of the hand of the subject; and the controller isconfigured to use the following equation to determine the trunk angle ofthe subject performing the task:T=a+b*f(H)+c*f(V)+d*f(w), where: a, b, c, and d are constants; H is avalue of the horizontal location of the hand of the subject performingthe task; V is a value of the vertical location of the hand of theperforming the task; w is a value of the width dimension of the subjectperforming the task; and T is a value of a trunk angle of the subjectperforming the task.
 8. The system of claim 1, wherein: the data relatedto the subject performing the task includes video data; and thecontroller is configured to determine the one or more dimensions of thesubject performing the task using pixel information from the video data.9. The system of claim 8, wherein the controller is configured toautomatically determine one or both of the trunk angle of the subjectperforming the task and the one or more values of trunk kinematics ofthe subject performing the task in real time during playback of thevideo data.
 10. The system of claim 1, wherein the trunk angle is one ofa trunk flexion angle and a spine flexion angle.
 11. A computer readablemedium having stored thereon in a non-transitory state a program codefor use by a computing device, the program code causing the computingdevice to execute a method for determining one or both of a trunk angleof a subject and trunk kinematics of the subject comprising: obtainingdata related to the subject performing a task; determining a value foreach of one or more dimensions of the subject performing the task basedon the data; determining a location of a hand of the subject performingthe task based on the data; and determining one or both of the trunkangle of the subject performing the task and one or more values of trunkkinematics of the subject performing the task based on the value for atleast one dimension of the one or more dimensions of the subjectperforming the task and the location of the hand of the subjectperforming the task.
 12. The computer readable medium of claim 11,wherein determining one or more values of trunk kinematics of thesubject performing the task includes determining a velocity of movementof a trunk of the subject performing the task.
 13. The computer readablemedium of claim 11, wherein determining one or more values of trunkkinematics of the subject performing the task includes determining anacceleration of movement of a trunk of the subject performing the task.14. The computer readable medium of claim 11, wherein the one or moredimensions of the subject performing the task includes one or more of aheight dimension of the subject and a width dimension of the subject.15. The computer readable medium of claim 11, wherein the location ofthe hand of the subject performing the task includes a horizontallocation of the hand of the subject and a vertical location of the handof the subject.
 16. A tracking system comprising: a processor; memory incommunication with the processor, the memory includes instructionsexecutable by the processor to: analyze pixel information in a video ofa subject performing a task; determine a value for each of one or moredimensions of the subject in a frame from the video based on the pixelinformation; and determine a trunk angle of the subject in the framebased on the value for at least one dimension of the one or moredimensions of the subject in the frame.
 17. The system of claim 16,wherein the memory includes further instructions executable by theprocessor to automatically determine trunk angles of the subject in realtime during playback of the video.
 18. The system of claim 16, whereinthe memory includes further instructions executable by the processor todetermine one or both of a velocity of movement of a trunk of thesubject over a plurality of frames from the video and an acceleration ofmovement of the trunk of the subject over a plurality of frames from thevideo.
 19. The system of claim 16, wherein the one or more dimensions ofthe subject in the frame includes one or more of a height dimension ofthe subject and a width dimension of the subject.
 20. The system ofclaim 16, wherein the memory includes further instructions executable bythe processor to: identify a ghost effect in the frame, the ghost effecthaving a location in the frame; determine a location of a hand of thesubject in the frame based on the location of the ghost effect;determine extreme-most pixels in a width dimension of the subject in theframe; assign a distance between the extreme-most pixels as a value ofthe width dimension; and determine the trunk angle of the subject in theframe based on the value of the width dimension of the subject in theframe and the location of the hand of the subject in the frame.